" Input: {
data_uoa - experiment data UOA (can have wildcards)
(repo_uoa) - experiment repo UOA (can have wildcards)
(experiment_repo_uoa) - use it, if repository is remote
(remote_repo_uoa) - if repo above is remote, use this repo on remote machine
(module_uoa)
(tags) - search by tags
(point) - point (or skip, if there is only one), can be of format UID-
(subpoint) - subpoint (or skip, if there is only one)
(repetitions) - statistical repetitions (default=4)
(pipeline_update) - customize pipeline with this dict (useful to update already prepared pipeline from file)
(dims_to_check) or (dims) - list of dimensions to check, can have wildcards (if empty, check all);
alternatively can use a string (useful for CMD)
(end_of_dims_to_check) - list of endings of dimensions to compare ...
(threshold_to_compare) - % threshold to decide whether dim is different or not
(record_original_flat_json) - file to record flat dict with original results
(record_reproduced_flat_json) - file to record flag dict with reproduced results
(skip) - if 'yes', do not perform comparison
==============================
If one would like to compare results with another point (for example, to check speedups
rather than just reproducing a given point)
(compare_data_uoa)
(compare_repo_uoa)
(compare_point)
(compare_subpoint)
==============================
Prunning (reducing complexity)
(prune) - if 'yes', replay and prune choices!
(reduce) - the same as above
(reduce_bug) - reduce choices to localize bug (pipeline fail)
(prune_md5) - if 'yes' and compilation is used in a pipeline (workflow), check if binary MD5 doesn't change
(prune_invert) - if 'yes', prune all (switch off even unused - useful for collaborative machine learning)
(prune_print_keys) - list of keys from flat dict to print during pruning (to monitor characteristics, for example)
(prune_invert_do_not_remove_key) - if 'yes', keep both on and off keys (to know exact solution)
(prune_conditions) - conditions on results (see "ck check math.conditions --help")
(condition_objective) - which objective to use for characteristics (#min, #max, #exp, ...)
(solutions) - prune first solution
==============================
(record_uoa) - (data UOA or CID where module_uoa ignored!) explicitly record to this entry
(record_repo) - (repo UOA) explicitly select this repo to record
==============================
Some productivity keys specifically for autotuning pipeline (ck-autotuning repo):
(local_platform) or (local) - if 'yes', use parameters of a local platform (to retarget experiment)
(skip_target) - do not select target machines (to customize via host_os/target_os/device_id)
(no_deps) - if 'yes', do not reuse deps
(skip_clean_after) - if 'yes', do not clean program pipeline after execution
(keeping low level scripts in tmp directory for low-level debugging)
(all) - print all comparisons of keys (not only when there is a difference)
(skip_scenario_keys) - skip scenario detection (otherwise get keys from the detected module)
}
Output: {
return - return code = 0, if successful
> 0, if error
(error) - error text if return > 0
output of a given pipeline
different_dims - list of different dimensions
}
"